6 results
PP54 Machine Learning For Accelerating Screening In Literature Reviews
- Mary Chappell, Mary Edwards, Deborah Watkins, Christopher Marshall, Lavinia Ferrante di Ruffano, Anita Fitzgerald, Sara Graziadio
-
- Journal:
- International Journal of Technology Assessment in Health Care / Volume 39 / Issue S1 / December 2023
- Published online by Cambridge University Press:
- 14 December 2023, p. S67
-
- Article
-
- You have access Access
- Export citation
-
Introduction
Systematic reviews are important for informing decision-making and primary research, but they can be time consuming and costly. With the advent of machine learning, there is an opportunity to accelerate the review process in study screening. We aimed to understand the literature to make decisions about the use of machine learning for screening in our review workflow.
MethodsA pragmatic literature review of PubMed to obtain studies evaluating the accuracy of publicly available machine learning screening tools. A single reviewer used ‘snowballing’ searches to identify studies reporting accuracy data and extracted the sensitivity (ability to correctly identify included studies for a review) and specificity, or workload saved (ability to correctly exclude irrelevant studies).
ResultsTen tools (AbstractR, ASReview Lab, Cochrane RCT classifier, Concept encoder, Dpedia, DistillerAI, Rayyan, Research Screener, Robot Analyst, SWIFT-active screener) were evaluated in a total of 16 studies. Fourteen studies were single arm where, although compared with a reference standard (predominantly single reviewer screening), there was no other comparator. Two studies were comparative, where tools were compared with other tools as well as a reference standard. All tools ranked records by probability of inclusion and either (i) applied a cut-point to exclude records or (ii) were used to rank and re-rank records during screening iterations, with screening continuing until most relevant records were obtained. The accuracy of tools varied widely between different studies and review projects. When used in method (ii), at 95 percent to 100 percent sensitivity, tools achieved workload savings of between 7 percent and 99 percent. It was unclear whether evaluations were conducted independent of tool developers.
ConclusionsEvaluations suggest the potential for tools to correctly classify studies in screening. However, conclusions are limited since (i) tool accuracy is generally not compared with dual reviewer screening and (ii) the literature lacks comparative studies and, because of between-study heterogeneity, it is not possible to robustly determine the accuracy of tools compared with each other. Independent evaluations are needed.
The contribution of depressive ‘disorder characteristics’ to determinations of prognosis for adults with depression: an individual patient data meta-analysis
- Joshua E. J. Buckman, Rob Saunders, Zachary D. Cohen, Phoebe Barnett, Katherine Clarke, Gareth Ambler, Robert J. DeRubeis, Simon Gilbody, Steven D. Hollon, Tony Kendrick, Edward Watkins, Nicola Wiles, David Kessler, David Richards, Deborah Sharp, Sally Brabyn, Elizabeth Littlewood, Chris Salisbury, Ian R. White, Glyn Lewis, Stephen Pilling
-
- Journal:
- Psychological Medicine / Volume 51 / Issue 7 / May 2021
- Published online by Cambridge University Press:
- 14 April 2021, pp. 1068-1081
-
- Article
-
- You have access Access
- Open access
- HTML
- Export citation
-
Background
This study aimed to investigate general factors associated with prognosis regardless of the type of treatment received, for adults with depression in primary care.
MethodsWe searched Medline, Embase, PsycINFO and Cochrane Central (inception to 12/01/2020) for RCTs that included the most commonly used comprehensive measure of depressive and anxiety disorder symptoms and diagnoses, in primary care depression RCTs (the Revised Clinical Interview Schedule: CIS-R). Two-stage random-effects meta-analyses were conducted.
ResultsTwelve (n = 6024) of thirteen eligible studies (n = 6175) provided individual patient data. There was a 31% (95%CI: 25 to 37) difference in depressive symptoms at 3–4 months per standard deviation increase in baseline depressive symptoms. Four additional factors: the duration of anxiety; duration of depression; comorbid panic disorder; and a history of antidepressant treatment were also independently associated with poorer prognosis. There was evidence that the difference in prognosis when these factors were combined could be of clinical importance. Adding these variables improved the amount of variance explained in 3–4 month depressive symptoms from 16% using depressive symptom severity alone to 27%. Risk of bias (assessed with QUIPS) was low in all studies and quality (assessed with GRADE) was high. Sensitivity analyses did not alter our conclusions.
ConclusionsWhen adults seek treatment for depression clinicians should routinely assess for the duration of anxiety, duration of depression, comorbid panic disorder, and a history of antidepressant treatment alongside depressive symptom severity. This could provide clinicians and patients with useful and desired information to elucidate prognosis and aid the clinical management of depression.
VP172 Clinical Effectiveness Of A Predictive Risk Model In Primary Care
- Helen Snooks, Alison Porter, Mark Kingston, Alan Watkins, Hayley Hutchings, Shirley Whitman, Jan Davies, Bridie Evans, Kerry Bailey-Jones, Deborah Burge-Jones, Jeremy Dale, Deborah Fitzsimmons, Martin Heaven, Helen Howson, Gareth John, Leo Lewis, Ceri Philips, Bernadette Sewell, Victoria Williams, Ian Russell
-
- Journal:
- International Journal of Technology Assessment in Health Care / Volume 33 / Issue S1 / 2017
- Published online by Cambridge University Press:
- 12 January 2018, p. 229
-
- Article
-
- You have access Access
- Export citation
-
INTRODUCTION:
New approaches are needed to safely reduce emergency admissions to hospital by targeting interventions effectively in primary care. A predictive risk stratification tool (PRISM) identifies each registered patient's risk of an emergency admission in the following year, allowing practitioners to identify and manage those at higher risk. We evaluated the introduction of PRISM in primary care in one area of the United Kingdom, assessing its impact on emergency admissions and other service use.
METHODS:We conducted a randomized stepped wedge trial with cluster-defined control and intervention phases, and participant-level anonymized linked outcomes. PRISM was implemented in eleven primary care practice clusters (total thirty-two practices) over a year from March 2013. We analyzed routine linked data outcomes for 18 months.
RESULTS:We included outcomes for 230,099 registered patients, assigned to ranked risk groups.
Overall, the rate of emergency admissions was higher in the intervention phase than in the control phase: adjusted difference in number of emergency admissions per participant per year at risk, delta = .011 (95 percent Confidence Interval, CI .010, .013). Patients in the intervention phase spent more days in hospital per year: adjusted delta = .029 (95 percent CI .026, .031). Both effects were consistent across risk groups.
Primary care activity increased in the intervention phase overall delta = .011 (95 percent CI .007, .014), except for the two highest risk groups which showed a decrease in the number of days with recorded activity.
CONCLUSIONS:Introduction of a predictive risk model in primary care was associated with increased emergency episodes across the general practice population and at each risk level, in contrast to the intended purpose of the model. Future evaluation work could assess the impact of targeting of different services to patients across different levels of risk, rather than the current policy focus on those at highest risk.
VP132 Cost Effectiveness Of A Predictive Risk Model In Primary Care
- Helen Snooks, Alison Porter, Mark Kingston, Bridie Evans, Deborah Burge-Jones, Jan Davies, Hayley Hutchings, Alan Watkins, Shirley Whitman, Bernadette Sewell, Kerry Bailey-Jones, Jeremy Dale, Deborah Fitzsimmons, Jane Harrison, Martin Heaven, Gareth John, Leo Lewis, Ceri Philips, Victoria Williams, Daniel Warm, Ian Russell
-
- Journal:
- International Journal of Technology Assessment in Health Care / Volume 33 / Issue S1 / 2017
- Published online by Cambridge University Press:
- 12 January 2018, pp. 209-210
-
- Article
-
- You have access Access
- Export citation
-
INTRODUCTION:
Emergency admissions to hospital are a major financial burden on health services. In one area of the United Kingdom (UK), we evaluated a predictive risk stratification tool (PRISM) designed to support primary care practitioners to identify and manage patients at high risk of admission. We assessed the costs of implementing PRISM and its impact on health services costs. At the same time as the study, but independent of it, an incentive payment (‘QOF’) was introduced to encourage primary care practitioners to identify high risk patients and manage their care.
METHODS:We conducted a randomized stepped wedge trial in thirty-two practices, with cluster-defined control and intervention phases, and participant-level anonymized linked outcomes. We analysed routine linked data on patient outcomes for 18 months (February 2013 – September 2014). We assigned standard unit costs in pound sterling to the resources utilized by each patient. Cost differences between the two study phases were used in conjunction with differences in the primary outcome (emergency admissions) to undertake a cost-effectiveness analysis.
RESULTS:We included outcomes for 230,099 registered patients. We estimated a PRISM implementation cost of GBP0.12 per patient per year.
Costs of emergency department attendances, outpatient visits, emergency and elective admissions to hospital, and general practice activity were higher per patient per year in the intervention phase than control phase (adjusted δ = GBP76, 95 percent Confidence Interval, CI GBP46, GBP106), an effect that was consistent and generally increased with risk level.
CONCLUSIONS:Despite low reported use of PRISM, it was associated with increased healthcare expenditure. This effect was unexpected and in the opposite direction to that intended. We cannot disentangle the effects of introducing the PRISM tool from those of imposing the QOF targets; however, since across the UK predictive risk stratification tools for emergency admissions have been introduced alongside incentives to focus on patients at risk, we believe that our findings are generalizable.
OP75 Implementing Risk Stratification In Primary Care: A Qualitative Study
- Alison Porter, Helen Snooks, Mark Kingston, Jan Davies, Hayley Hutchings, Shirley Whitman, Alan Watkins, Bridie Evans, Kerry Bailey-Jones, Deborah Burge-Jones, Jeremy Dale, Deborah Fitzsimmons, Jane Harrison, Helen Howson, Martin Heaven, Gareth John, Leo Lewis, Ceri Philips, Bernadette Sewell, Daniel Warm, Victoria Williams, Ian Russell
-
- Journal:
- International Journal of Technology Assessment in Health Care / Volume 33 / Issue S1 / 2017
- Published online by Cambridge University Press:
- 12 January 2018, pp. 34-35
-
- Article
-
- You have access Access
- Export citation
-
INTRODUCTION:
A predictive risk stratification tool (PRISM) to estimate a patient's risk of an emergency hospital admission in the following year was trialled in general practice in an area of the United Kingdom. PRISM's introduction coincided with a new incentive payment (‘QOF’) in the regional contract for family doctors to identify and manage the care of people at high risk of emergency hospital admission.
METHODS:Alongside the trial, we carried out a complementary qualitative study of processes of change associated with PRISM's implementation. We aimed to describe how PRISM was understood, communicated, adopted, and used by practitioners, managers, local commissioners and policy makers. We gathered data through focus groups, interviews and questionnaires at three time points (baseline, mid-trial and end-trial). We analyzed data thematically, informed by Normalisation Process Theory (1).
RESULTS:All groups showed high awareness of PRISM, but raised concerns about whether it could identify patients not yet known, and about whether there were sufficient community-based services to respond to care needs identified. All practices reported using PRISM to fulfil their QOF targets, but after the QOF reporting period ended, only two practices continued to use it. Family doctors said PRISM changed their awareness of patients and focused them on targeting the highest-risk patients, though they were uncertain about the potential for positive impact on this group.
CONCLUSIONS:Though external factors supported its uptake in the short term, with a focus on the highest risk patients, PRISM did not become a sustained part of normal practice for primary care practitioners.
Contributors
-
- By Aakash Agarwala, Linda S. Aglio, Rae M. Allain, Paul D. Allen, Houman Amirfarzan, Yasodananda Kumar Areti, Amit Asopa, Edwin G. Avery, Patricia R. Bachiller, Angela M. Bader, Rana Badr, Sibinka Bajic, David J. Baker, Sheila R. Barnett, Rena Beckerly, Lorenzo Berra, Walter Bethune, Sascha S. Beutler, Tarun Bhalla, Edward A. Bittner, Jonathan D. Bloom, Alina V. Bodas, Lina M. Bolanos-Diaz, Ruma R. Bose, Jan Boublik, John P. Broadnax, Jason C. Brookman, Meredith R. Brooks, Roland Brusseau, Ethan O. Bryson, Linda A. Bulich, Kenji Butterfield, William R. Camann, Denise M. Chan, Theresa S. Chang, Jonathan E. Charnin, Mark Chrostowski, Fred Cobey, Adam B. Collins, Mercedes A. Concepcion, Christopher W. Connor, Bronwyn Cooper, Jeffrey B. Cooper, Martha Cordoba-Amorocho, Stephen B. Corn, Darin J. Correll, Gregory J. Crosby, Lisa J. Crossley, Deborah J. Culley, Tomas Cvrk, Michael N. D'Ambra, Michael Decker, Daniel F. Dedrick, Mark Dershwitz, Francis X. Dillon, Pradeep Dinakar, Alimorad G. Djalali, D. John Doyle, Lambertus Drop, Ian F. Dunn, Theodore E. Dushane, Sunil Eappen, Thomas Edrich, Jesse M. Ehrenfeld, Jason M. Erlich, Lucinda L. Everett, Elliott S. Farber, Khaldoun Faris, Eddy M. Feliz, Massimo Ferrigno, Richard S. Field, Michael G. Fitzsimons, Hugh L. Flanagan Jr., Vladimir Formanek, Amanda A. Fox, John A. Fox, Gyorgy Frendl, Tanja S. Frey, Samuel M. Galvagno Jr., Edward R. Garcia, Jonathan D. Gates, Cosmin Gauran, Brian J. Gelfand, Simon Gelman, Alexander C. Gerhart, Peter Gerner, Omid Ghalambor, Christopher J. Gilligan, Christian D. Gonzalez, Noah E. Gordon, William B. Gormley, Thomas J. Graetz, Wendy L. Gross, Amit Gupta, James P. Hardy, Seetharaman Hariharan, Miriam Harnett, Philip M. Hartigan, Joaquim M. Havens, Bishr Haydar, Stephen O. Heard, James L. Helstrom, David L. Hepner, McCallum R. Hoyt, Robert N. Jamison, Karinne Jervis, Stephanie B. Jones, Swaminathan Karthik, Richard M. Kaufman, Shubjeet Kaur, Lee A. Kearse Jr., John C. Keel, Scott D. Kelley, Albert H. Kim, Amy L. Kim, Grace Y. Kim, Robert J. Klickovich, Robert M. Knapp, Bhavani S. Kodali, Rahul Koka, Alina Lazar, Laura H. Leduc, Stanley Leeson, Lisa R. Leffert, Scott A. LeGrand, Patricio Leyton, J. Lance Lichtor, John Lin, Alvaro A. Macias, Karan Madan, Sohail K. Mahboobi, Devi Mahendran, Christine Mai, Sayeed Malek, S. Rao Mallampati, Thomas J. Mancuso, Ramon Martin, Matthew C. Martinez, J. A. Jeevendra Martyn, Kai Matthes, Tommaso Mauri, Mary Ellen McCann, Shannon S. McKenna, Dennis J. McNicholl, Abdel-Kader Mehio, Thor C. Milland, Tonya L. K. Miller, John D. Mitchell, K. Annette Mizuguchi, Naila Moghul, David R. Moss, Ross J. Musumeci, Naveen Nathan, Ju-Mei Ng, Liem C. Nguyen, Ervant Nishanian, Martina Nowak, Ala Nozari, Michael Nurok, Arti Ori, Rafael A. Ortega, Amy J. Ortman, David Oxman, Arvind Palanisamy, Carlo Pancaro, Lisbeth Lopez Pappas, Benjamin Parish, Samuel Park, Deborah S. Pederson, Beverly K. Philip, James H. Philip, Silvia Pivi, Stephen D. Pratt, Douglas E. Raines, Stephen L. Ratcliff, James P. Rathmell, J. Taylor Reed, Elizabeth M. Rickerson, Selwyn O. Rogers Jr., Thomas M. Romanelli, William H. Rosenblatt, Carl E. Rosow, Edgar L. Ross, J. Victor Ryckman, Mônica M. Sá Rêgo, Nicholas Sadovnikoff, Warren S. Sandberg, Annette Y. Schure, B. Scott Segal, Navil F. Sethna, Swapneel K. Shah, Shaheen F. Shaikh, Fred E. Shapiro, Torin D. Shear, Prem S. Shekar, Stanton K. Shernan, Naomi Shimizu, Douglas C. Shook, Kamal K. Sikka, Pankaj K. Sikka, David A. Silver, Jeffrey H. Silverstein, Emily A. Singer, Ken Solt, Spiro G. Spanakis, Wolfgang Steudel, Matthias Stopfkuchen-Evans, Michael P. Storey, Gary R. Strichartz, Balachundhar Subramaniam, Wariya Sukhupragarn, John Summers, Shine Sun, Eswar Sundar, Sugantha Sundar, Neelakantan Sunder, Faraz Syed, Usha B. Tedrow, Nelson L. Thaemert, George P. Topulos, Lawrence C. Tsen, Richard D. Urman, Charles A. Vacanti, Francis X. Vacanti, Joshua C. Vacanti, Assia Valovska, Ivan T. Valovski, Mary Ann Vann, Susan Vassallo, Anasuya Vasudevan, Kamen V. Vlassakov, Gian Paolo Volpato, Essi M. Vulli, J. Matthias Walz, Jingping Wang, James F. Watkins, Maxwell Weinmann, Sharon L. Wetherall, Mallory Williams, Sarah H. Wiser, Zhiling Xiong, Warren M. Zapol, Jie Zhou
- Edited by Charles Vacanti, Scott Segal, Pankaj Sikka, Richard Urman
-
- Book:
- Essential Clinical Anesthesia
- Published online:
- 05 January 2012
- Print publication:
- 11 July 2011, pp xv-xxviii
-
- Chapter
- Export citation